On a Multivariate Eigenvalue Problem: Ii. Global Solutions and the Gauss-seidel Method

Optimizing correlation between sets of variables is an important task in many areas of applications. The stationary points where the optimal correlation occurs are exactly the solutions to the multivariate eigenvalue problem (MEP). For decades, the Horst algorithm which later was recognized as an aggregated Jacobi-type power method has been regarded as an effective numerical scheme for tackling the MEP. Following the innate iterative structure, a generalization to the Gauss-Seidel formulation has been proposed as a natural improvement on the power method for the MEP. To this date, however, no convergence proof for the Gauss-Seidel algorithm has been established. Most disappointingly, neither algorithm can guarantee convergence to a global maximizer, which would have significant impact on applications. To these regards, this paper contains two contributions. First, some distinctive properties of the global maxima are characterized, which makes it possible to propose an effective starting point strategy for obtaining a global maximizer. Secondly, monotone convergence of the Gauss-Seidel algorithm to a solution for the MEP is proved. Numerical testing of the Gauss-Seidel algorithm combined with the starting point strategy suggests a superior performance to the conventional Horst-Jacobi algorithm without the starting point strategy.